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DETAILED ACTION 

Drawings 

1 . The drawings are objected to because of multiple informalities. 

• Reference 300E is described in figure 12 on page 27, line 9, but is not 
actually labeled in figure 12. 

• In figures 14 and 1 5 the word "Exteranous" is used. It would appear that 
the word intended was "Extraneous". 

• In figure 15, box 1320 has the word pixels misspelled ("pexels") 

• In figure 19, box 1714 has an extra line of text at the end ("MORE ROI") 
that appears to be an incomplete deletion. 

• In figure 19, decision block 1746 appears to have the "YES" and "NO" 
labels reversed. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply 
to the Office action to avoid abandonment of the application. Any amended replacement 
drawing sheet should include all of the figures appearing on the immediate prior version 
of the sheet, even if only one figure is being amended. The figure or figure number of an 
amended drawing should not be labeled as "amended." If a drawing figure is to be 
canceled, the appropriate figure must be removed from the replacement sheet, and 
where necessary, the remaining figures must be renumbered and appropriate changes 
made to the brief description of the several views of the drawings for consistency. 
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Additional replacement sheets may be necessary to show the renumbering of the 
remaining figures. Each drawing sheet submitted after the filing date of an application 
must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" 
pursuant to 37 CFR 1.121 (d). If the changes are not accepted by the examiner, the 
applicant will be notified and informed of any required corrective action in the next Office 
action. The objection to the drawings will not be held in abeyance. 

Specification 

The following quotation of 37 CFR § 1 .72(b) is the basis of objection: 

(b) A brief abstract of the technical disclosure in the specification must commence 
on a separate sheet, preferably following the claims, under the heading "Abstract" 
or "Abstract of the Disclosure." The sheet or sheets presenting the abstract may 
not include other parts of the application or other material. The abstract in an 
application filed under 35 U.S.C. 1 1 1 may not exceed 150 words in length. The 
purpose of the abstract is to enable the United States Patent and Trademark 
Office and the public generally to determine quickly from a cursory inspection the 
nature and gist of the technical disclosure. 

The abstract of the disclosure is objected to because of undue length. Correction is 
required. See MPEP § 608.01(b). 

2. The disclosure is objected to because of the following informalities: 

• Pages 19-22 make reference to several "pseudo-images" such as "600-A" 
and "700-B". These appear to be uncorrected references to an earlier 
drawing set and are now labeled as figures (e.g. "Fig. 6A") in the current 
drawing set. The specification should be updated to match the drawings. 

• On page 21 , line 1 1 , the reference to the template 700-B should have 
referenced 700-A in the original drawing and Fig. 7A in the revised 
drawings (it was mis-referenced before the drawings were changed). 

Appropriate correction is required. 
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Claim Objections 

3. Claim 22 is objected to because of the following informalities: 

Claim 22 is a method claim depending from an apparatus claim. From context it would 

appear that claim 22 was intended to be another apparatus claim and for the purposes 

of the remainder of this action claim 22 will be presumed to read as: 

♦ 

The machine vision inspection system of Claim 18, wherein the extraneous feature 
identifying portion comprises predetermined extraneous feature pixel identification 
operations, and the user interface includes features usable by an operator to determine 
at least one of S1 ) at least one subset of the predetermined extraneous feature pixel 
identification operations, to be applied to the workpiece image, and S2) at least one 
parameter that governs a predetermined extraneous feature pixel identification 
operation to be applied to a workpiece image, and the extraneous feature identifying 
portion identifies extraneous feature pixels based on the operator determination of at 
least one of S 1 ) and S2). 
Appropriate correction is required. 
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Claim Objections - 37 CFR 1.75(a) 

4. The following is a quotation of 37 CFR 1 .75(a): 

The specification must conclude with a claim particularly pointing out and 
distinctly claiming the subject matter which the applicant regards as his invention 
or discovery. 

5. Claims 1 1 and 21 are objected to under 37 CFR 1 .75(a), as failing to particularly 
point out and distinctly claim the subject matter which applicant regards as his invention 
or discovery. 

Regarding claim 1 1 , the phrase "appropriate manner" is vague and indefinite. 

Regarding claim 21 , It would seem that this claim is intended to associate the ROI 
(defined in claim 18) to a widget associated with the video tool (also defined in claim 18) 
and additionally associate the mode control or operating status feature described in 
claim 20 to another widget associated with the same tool. The way it is currently worded 
is quite repetitive and if a) and b) when it first appears is intended to reference the 
definitions in claim 20, the subsequent use of a) and b) at the end of claim 21 are 
inconsistent in that they appear to flip the definitions (i.e. what was a) is now defined as 
b) and vice versa). 
Appropriate correction is required. 
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Claim 1 7 is objected to under 37 CFR 1 .75(c), as being of improper dependent 
form for failing to further limit the subject matter of a previous claim. Applicant is 
required to cancel the claim(s), or amend the claim(s) to place the claim(s) in proper 
dependent form, or rewrite the claim(s) in independent form. Claim 17 fails to further 
limit claim 16. That is, claim 16 is a method, and claim 17 does not advance or 
otherwise further limit any steps in the method and it therefore not given any patentable 
weight. 

Claim Objections - 37 CFR 1. 75(d)(1) 

6. The following is a quotation of 37 CFR 1 .75(d)(1 ): 

The claim or claims must conform to the invention as set forth in the remainder of the specification and 
the terms and phrases used in the claims must find clear support or antecedent basis in the description 
so that the meaning of the terms in the claims may be ascertainable by reference to the description. 

7. Claim 8 is objected to under 37 CFR 1 .75(d)(1 ), as failing to conform to the 
invention as set forth in the remainder of the specification. The "suspend" feature 
referred to in this claim is not disclosed in the specification. Appropriate correction is 
required. It is suggested that the applicant should add this element back to the 
specification without adding new matter. 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claims 1-3, 6-10, 12-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Chau et al. (US 5,879,698 A) and Silver et al. (US 5,481,712 A). 

Regarding claim 1, Chau teaches an apparatus and method: 

acquiring an image including an extraneous feature ("a sample image is 
acquired", Chau, column 2, line 36); 

identifying extraneous feature pixels at least in the region of interest; and 
performing the one or more operations associated with the video tool, 
excluding the identified extraneous feature pixels from at least one 
operation associated with the video tool, at least in the region of interest, 
("a final image having only macro defects, if any, is produced", Chau, 
column 2, line 45). 



Chau does not teach: 

defining a region of interest associated with a video tool provided by the 
user interface, the video tool having one or more associated operations; 



Application/Control Number: 10/808,948 Page 8 

Art Unit: 2609 

Silver teaches defining a region of interest associated with a video tool provided 
by the user interface, the video tool having one or more associated operations 
("Measurements performed on the digital image can be performed on selected regions 
of interest", Silver, column 7, line 40, "the graphical input element 212 superimposes a 
manipulate marquee/box over the candidate image, thereby permitting the operator to 
designate the region of interest", Silver, column 5, line 38) and "the machine vision tools 
include machine vision analyses, geometric constructions built from them, and/or 
automation controls", Silver, column 7, line 34) 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to combine the vision tools and associated ROI capability of the vision 
system of Silver with the edge removal method of Chau. ROIs to specify the portion of 
the image to operate on ("designation of a portion of image in which that tool is to 
operate", Silver, Column 5, line 37). Additionally, the use of ROIs would reduce the 
overall processing load of the system improving system performance. The use of the 
video tools to provide the processing capability that Chau indicates is needed to identify 
defects ("the automated image processing can be accomplished using a general 
purpose computer with appropriate program routines or by using a specialized image 
processing system with other appropriate program routines", Chau, column 2, line 67 
and Chau, steps 260 and 270 in Fig. 2) 
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Regarding claim 2, Chau teaches the extraneous feature comprises at least 
one of a) a grid-like feature, and b) a grid-like feature of a flat panel display screen mask 
(Fig. 3, Chau) 

Regarding claim 3, Chau describes image data corresponding to pixels in the 
region of interest that are not the identified extraneous feature pixels are not modified 
before performing the one or more operations associated with the video tool ("a 
difference image created where edge enhanced scattering effects present in both the 
template and the enhanced sample image are essentially eliminated" Chau, column 2, 
line 39). 

Regarding claim 6, the combination of Chau and Silver described above does 
teach that the extraneous feature pixels are identified only in the region of interest in 
that the ROI is used to specify the portion of the image to operate on. 

Regarding claims 7, 12 and 20, the system disclosed by Chau includes a 
mode of operation wherein operations usable to identify extraneous feature pixels are 
activated, and the user interface comprises at least one control widget usable to 
activate the mode of operation. Chau's describes his system as having a "training 
phase" (Chau, column 3, line 57) and a "detection phase" (Chau, column 4, line 39). 
Such a system will by necessity have control elements to activate each mode (phase) 
and to indicate it's current mode. 
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Regarding claim 8, the combination of Chau and Silver above does provide a 
machine vision system with user interface as required above, the combination does not 
expressly teach a user interface that includes at least one feature that is usable to 
suspend the mode of operation of the machine vision inspection system wherein 
operations usable to identify extraneous feature pixels are activated. 

However, it is well known in the art to provide software running on a computer 
based systems with the ability to pause or terminate processing; e.g., the ability to close 
the software (Official Notice). It is also well known to provide the operator with the ability 
to power down the computer, even when software is running and will be interrupted 
(Official Notice). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to provide the system of the Chau and Silver combination, with the well 
known ability to suspend operations as described above to pause or stop the software 
so that other software operations can be performed, or to power down or reset the 
computer system. 

Regarding claims 9 and 21, The vision tools of Silver, included in the combination 
above teach an apparatus and method where the region of interest defining portion is 
usable to define the respective region of interest based on a graphical operator- 
configurable region of interest indicating widget corresponding to the respective video 
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tool (Silver, Figure 2, where reference 216 is a marquee selection box for defining an 
ROI), and the at least one feature that is usable for at least one of a) and b) is 
associated with the respective video tool and the at least one feature comprises at least 
one of a) a mode status indicating feature and b) a mode control widget (Silver, Figure 
2, reference 204 shows the logical structure and flow for defining and adjusting 
parameters associated with a graphical vision tool). 

Regarding claim 10, Chau teaches using thresholding techniques to identify 
regions corresponding to extraneous feature pixels ("thresholded difference image 240 
is essentially an array of pixels where macro defects are represented as pixels", Chau, 
column 5, line 8) and morphology techniques to filter anomalous portions of the 
boundaries of the identified regions ("The morphological operation of optional Step 250 
filters image 240 to eliminate clusters (contiguous groups) of pixels that do not 
represent true macro defects, while leaving intact those that do" Chau, column 5, line 
21). 

Regarding claims 13 and 22, the combination of Chau and Silver described 
above teach the apparatus and method of Claims 1 and 18, wherein the extraneous 
feature identifying portion comprises predetermined extraneous feature pixel 
identification operations (described above in the paragraphs dealing with claims 1 and 
18), and the user interface includes features usable by an operator to determine at least 
one of S1 ) at least one subset of the predetermined extraneous feature pixel 
identification operations, to be applied to the workpiece image, and S2) at least one 
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parameter that governs a predetermined extraneous feature pixel identification 
operation to be applied to a workpiece image (Silver, Figure 2, reference 204 shows the 
logical structure and flow for defining and adjusting parameters associated with a 
graphical vision tool), and the extraneous feature identifying portion identifies 
extraneous feature pixels based on the operator determination of at least one of S 1 ) 
and S2) 

Regarding claim 14, The combination of Chau and Silver described above teach 
the method of Claim 13, wherein identifying extraneous feature pixels at least in the 
region of interest comprises: identifying a set of extraneous feature pixels according to 
the operator selection of at least one of S 1) and S2) (as discussed above for claim 13); 
providing a display of the identified set of extraneous feature pixels superimposed at 
their proper locations on at least a portion of the original inspection image such that they 
are easily distinguished; and the operator approving the identified set of extraneous 
feature pixels based on the display, wherein the operator approval takes place prior to 
performing the one or more operations associated with the video tool ( Chau, Figure 1, 
Step 140 is a decision block where the operator may determine if additional references 
are needed and either stop or continue training, "if additional training is desired to 
account for acceptable process variations, branch 128 is followed and another 
reference image, Step 130, acquired.", Chau, column 4, line 12). 
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Regarding claim 15, the combination of Chau and Silver, in the vision tools of 
Silver, teach the method of Claim 14, wherein the method is performed during a 
training mode of operation of the machine vision inspection system and when the 
operator approval takes place prior to performing the one or more operations associated 
with the video tool, machine control instructions corresponding to the operator selection 
of at least one of S1 ) and S2) are in a part program for automatically inspecting the 
workpiece (Silver, Figure 2, references 230 and 234, graphical manipulation of video 
tools results in the creation and modification of executable code, and "Once the 
operator has specified the input parameters, the update element 230 generates 
modifies (sic) the stored program to include in the call to the requested machine vision 
tool, e.g., LINE FINDER, the designated input parameters." Silver, column 5, line 46). 

Regarding claim 16, Chau teaches a first set of operations used for identifying 
extraneous feature pixels at least in the region of interest are performed prior to 
beginning a second set of operations used for performing the one or more operations 
associated with the video tool (Fig. 2, Chau, elements 210 - 255 occur first and deal 
with identifying and removing extraneous edge features, elements 260 and 270 
correspond to defect analysis on non-extraneous pixels and occur afterwards). 

Regarding claim 17, wherein in the source code of a program used for 
controlling operations of the machine vision inspection system, program instructions for 
performing the first set of operations (identifying extraneous features) are not 
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interspersed with program instructions for performing the second set of operations 
(Operations associated with the video tool), Fig 2 in Chau, as described above, shows 
the structure of the program where the code identifying extraneous features is separate 
from the defect detection code. 

Regarding claim 18, the combination of Chau and Silver described above teach: 
a user interface usable to define a set of operations usable to inspect a workpiece 
image ("Thus, when the operator selects a machine vision tool (through a menu choice), 
the DS 320 sends a request via data pathway 318 to the VP 330 for its creation, records 
the information that results from that creation, and gives the tool a default name so it 
can be listed in the variables display." Silver, column 6, line 44); 
an image acquisition portion usable for acquiring a workpiece image that includes 
an extraneous feature (Chau, figure 2, Step 200) 

a region of interest defining portion usable to define a respective region of 

interest in the acquired image, 

the respective region of interest corresponding to a respective 
video tool of a type provided by the user interface, 
the video tool having one or more associated operations ("designation of a 
portion of image in which that tool is to operate", Silver, Column 5, line 37, 
and "the machine vision tools include machine vision analyses, geometric 
constructions built from them, and/or automation controls", Silver, column 
7, line 34); and 
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an extraneous feature identifying portion usable to identify extraneous 
feature pixels at least in the respective region of interest, 
wherein, 

when the extraneous feature identifying portion is used to identify 
extraneous feature pixels at least in the respective region of 
interest of the workpiece image, the machine vision inspection 
system is operable to perform the one or more 
operations associated with the respective video tool and exclude 
the identified extraneous feature pixels from at least one operation 
associated with the video tool (Chau, Fig. 3, where the sequence 
shown illustrates identifying extraneous features and removing 
them from the image to simplify detect detection and "a difference 
image created where edge enhanced scattering effects present in 
both the template and the enhanced sample image are essentially 
eliminated. Again, applying one or more automated image 
processing techniques, for example GTC thresholding, 
morphological transformations and blob analysis, macro defect 
detection can be further improved. In this manner, a final image 
having only macro defects, if any, is produced." Chau, column 2, 
line 37). 
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Regarding claim 19, the combination of Chau and Silver discloses the 
extraneous feature identifying portion is used to identify extraneous feature pixels 
included in the respective region of interest, the machine vision inspection system 
automatically excludes the identified extraneous feature pixels from at least one 
operation associated with the respective video tool (Chau, Figure 3, which illustrates an 
image to be analyzed for defects after extraneous features have been removed and A 
difference image is formed by comparing the edge enhanced reference and sample 
images. The difference image is evaluated using one or more automated image 
processing techniques such as thresholding, morphological transformations and blob 
analysis to identify macro defects." Chau, Abstract, line 6). 

10. Claims 4 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chau and Silver as applied to the claims above, and further in view of Petry III et al. (US 
6,859,923 A). 

Regarding claim 4, the combination of Chau and Silver does not describe 
identifying a first set of pixels estimated to correspond to the extraneous feature; 
determining a second set of pixels that comprise a buffer region adjacent to at 
least some of the pixels of the first set; and including pixels corresponding to the first 
and second sets in the identified extraneous feature pixels. 



Application/Control Number: 10/808,948 Page 17 

Art Unit: 2609 

Petry working in a related problem solving area of mark defect analysis describes 
"The mark background region 64 and the character background regions 70, less a 
padding factor two pixels wide around the ink, form the defect analysis masks for the 
backgrounds." Petry, column 6, line 56 and figure 3D. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the defect analysis mask technique of Petry, in particular the 
background mask, to accommodate variation in the edge enhanced scattering effects 
between the reference and sample images ("slight changes in the inked regions are 
measured when inspecting using the ink mask and not when inspecting with the 
background mask" Petry, column 6, line 50). 

Regarding claim 5, the above combination of Chau, Silver and Petry, in the 
defect analysis mask technique of Petry describes determining the second set of pixels 
comprises performing at least one dilation operation on the first set of pixels ("the ink 
masks are dilated by a padding factor and background masks are eroded by the same 
factor", Petry, column 6, line 48). 

Allowable Subject Matter 
1 1 . Claim 1 1 is objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 
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The following are statements of reasons for the indication of allowable subject 
matter in the claims listed above: 

Regarding claim 1 1 , the prior art of record does not teach characterizing a 
nominal geometric feature corresponding to at least one portion of the extraneous 
feature based on a apriori knowledge of the workpiece; or identifying pixels 
corresponding to the located nominal geometric feature as extraneous feature pixels. 

Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Thomas M. Redding whose telephone number is (571 ) 
270-1579. The examiner can normally be reached on Mon - Fri 7:30 am - 5:00 pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian P. Werner can be reached on (571 ) 272-7401 . The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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